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Abstract 

Image  processing  algorithms  enable  a  kinetic  kill  ve¬ 
hicle  (KKV)  with  an  IR  imaging  sensor  to  intercept  a  missile 
warhead  in  space.  The  algorithms  are  required  to  determine 
the  target  position  soon  enough  and  with  such  accuracy  that 
the  KKV  can  divert  itself  toward  a  successful  impact.  This 
is  affected  by  1)  target  brightness,  shape,  rotational  motion, 
and  associated  objects  such  as  debris,  2)  sensor  characteris¬ 
tics  including  aperture  size  and  signal-to-noise  ratios,  3)  the 
KKVs  divert  capability  and  IMU  accuracy,  and  4)  the  ac¬ 
quisition  range  and  closing  velocity  for  the  target  relative  to 
the  KKV.  Furthermore,  the  algorithms  are  constrained  to  run 
on  embedded  digital  signal  processing  (DSP)  hardware  with 
finite  throughput  and  memory  capacity. 

This  paper  presents  image  processing  algorithms  for 
KKVs  with  IR  imaging  sensors  and  describes  I )  the  funda¬ 
mental  requirement  for  target  position  estimates,  2 )  the  three 
potentially  most  dijficult  parts  of  the  problem  (detection, 
selection,  and  aimpoint),  3)  various  algorithms  that  can  be 
used,  and  4)  partitioning  the  algorithms  for  implementation 
in  multi-microprocessor  subsystems. 

1  Introduction 

The  KKV  mission  is  to  destroy  a  missile  warhead 
which  may  still  be  attached  to  the  launch  vehicle,  or  perhaps 
a  stage  of  that  vehicle,  or  else  the  warhead  may  be  a  detached 
re-entry  vehicle  (RV).  The  ability  of  a  KKV  system  to  de¬ 
fend  against  the  attack  is  dependent  upon  1)  the  surveillance 
system’s  ability  to  detect  a  toeat,  2)  the  accuracy  with  which 
ground  or  airborne  control  can  vector  the  boost  vehicle  to¬ 
wards  the  kill  point,  3)  the  boost  vehicle’s  deployment 
envelope,  4)  the  ICKV  thrusters’  ability  to  remove  remaining 
errors,  5)  the  guidance,  navigation  and  control  (GN&C)  sys¬ 
tem  accuracy,  and  6)  the  capabihty  of  the  KKV’s  sensor  and 
image  processing.  This  paper  considers  only  requirements 
for  topic  6). 

The  KKV  cannot,  in  general,  impact  a  warhead  at  an 
exact  point.  Errors  from  thrusters,  attitude  control,  and  com¬ 
putation  result  in  an  impact  some  distance  from  the  desired 
point.  The  distance  between  the  desired  and  actual  impact 
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points  will  be  referred  to  as  the  miss  distance.  The  allowable 
miss  distance,  which  defines  the  kill  region  on  the  target,  can 
be  extended  backwards  in  time  to  define  a  surface  that  would 
result  from  utilizing  the  maximum  KKV  capability.  If  the 
target  is  acquired  at  a  long  range,  then  the  divert  capability 
can  be  used  early  in  the  mission  to  remove  a  relatively  large 
initial  miss  distance  with  an  initial  lateral  delta  velocity  that 
is  integrated  over  the  remainder  of  the  mission.  Since  the 
surface  is  reduced  by  image  processing  errors,  the  errors 
should  cause  a  negligible  contribution  to  the  decrease  in  sur¬ 
face  size. 

The  image  processing  algorithms  must  determine  the 
target  line-of-sight  (LOS)  in  an  accurate  and  timely  manner 
so  that  the  KKV  can  divert  to  intercept.  If  the  target  is  ac¬ 
quired  sooner  and  if  the  image  processing  estimate  is 
accurate,  then  the  surface  defining  the  reachable  targets  is 
larger.  This  paper  describes  1)  the  requirements  for  image 
processing,  2)  acquisition,  discrimination,  and  aimpoint 
problems,  3)  algorithms  that  can  be  used,  and  4)  multi-mi¬ 
croprocessor  implementation  of  the  algorithms. 

2  Image  Processing  Algorithm  Requirements 

There  are  several  conflicting  requirements  for  achiev¬ 
ing  the  highest  probabihty  of  kill.  Chi  one  hand,  the  target 
should  be  selected  as  soon  as  possible  so  the  divert  capability 
can  be  used  early  in  the  mission  to  maximize  the  size  of  the 
surface  defining  the  reachable  kill  region  on  the  target.  On 
the  other  hand,  at  long  ranges  the  target  is  a  point  source.  If 
other  point  sources  such  as  debris  are  present,  target  selec¬ 
tion  must  be  delayed  until  there  is  a  hi^  degree  of 
confidence  that  the  correct  object  is  selected. 

Once  the  target  is  acquired,  the  image  processing  must 
estimate  the  LOS  direction  toward  a  particular  point  in  the 
target.  The  changes  in  that  direction  indicate  cross  range 
movement  of  the  target  relative  to  the  KKV  A  constant  bias 
in  the  LOS  estimate  does  not  affect  direction  changes,  so 
constant  bias  errors  can  be  ignored.  Other  errors  are  due  to 
1)  not  detecting  the  target  or  not  selecting  which  detected  ob¬ 
ject  is  the  target,  2)  estimating  the  position  of  the  selected 
target  while  it  is  still  a  point  source,  and  3)  determining 
which  point  in  a  target  image  is  the  desired  impact  point. 
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Figure  1.  Algorithm  phase  transitions. 


The  requirements  can  thus  be  summarized  as  follows. 
1)  Detect  all  objects  as  early  in  the  mission  as  possible,  2) 
Select  the  target  object  with  a  high  confidence  level  in  a 
timely  manner,  3)  Provide  accurate  aimpoint  LOS,  and  4) 
If  the  kill  point  on  the  warhead  is  different  from  the  centroid 
of  the  taiget  vehicle,  select  the  warhead  end  of  the  target,  as 
soon  as  it  can  be  determined  with  a  high  probability,  yet  al¬ 
lowing  sufficient  time  to  divert  to  the  kill  point. 

3  Detection.  Selection,  and  Aimpoint  Problems 

3.1  Detection  Of  Target  At  Long  Range 

Again  conflicting  requirements  must  be  resolved.  The 
kill  vehicle  must  be  sufficiently  small  that  it  can  be  boosted 
to  a  laige  intercept  volume  with  a  minimum  cost  which  con¬ 
flicts  with  the  need  for  a  large  aperture  telescope  to  collect 
sufficient  photons  from  a  distant  target.  Complicating  mat¬ 
ters  is  noise,  including  pattern,  white  and  one  over  /  (l/f) 
noise  [1],  imperfections  in  the  focal  plane  including  dead  de¬ 
tectors,  and  nonuniform  responsivity  of  the  detectors. 

It  is  well  known  that  signal-to-noise  ratio  (SNR)  is  in¬ 
creased  by  integration,  so  enhancing  SNR  is  an  obvious 
technique  for  increasing  detection  range.  On-focal  plane  in¬ 
tegration  must  be  accomplished  so  as  to  preclude  capacitor 
and  multiplexor  (MUX)  saturation,  and  to  accommodate 
available  analog-to-digital  converter  (ADC)  capability. 

3.2  Target  Selection 

Generally,  at  desired  acquisition  ranges  all  objects  in 
the  field-of-regard  (FOR)  are  point  sources.  The  taiget  must 
be  discriminated  from  booster  fragments  and  other  debris, 
decoys,  stars  and  other  space  objects  such  as  satellites,  etc. 
Closely  spaced  objects  (CSO)  must  be  either  resolved  or,  un¬ 
til  resolution  is  possible,  included  as  potential  targets. 

The  observable  data  for  selecting  the  taiget  is  intensity, 
intensity  rate,  and  apparent  motion.  Discriminants  derivable 
from  this  data  include  tumbling  or  coning  motion,  passive 
range  and  closing  rate  based  on  ground  based  radar  initial  es¬ 
timates.  Multi-wave  band  data  yields  temperature, 
temperature  rate,  and  emissivity-area  product.  Unfortunate¬ 
ly,  due  to  the  limited  telescope  aperture,  optical  aberrations, 
and  the  noise  in  the  readout,  the  taiget  image  is  blurred  and 
corrupted  with  noise.  Accurately  extraction  of  discriminants 
requires  additional  image  processing  to  provide  SNR  en¬ 
hancement. 
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If  external  target  information  is  available,  this  external 
data  must  be  fused  into  the  discrimination  process  to  opti¬ 
mize  correct  taiget  selection. 

3.3  Aimpoint  Selection 

If  the  target  is  a  separated  reentry  vehicle  (RV)  it  is  not 
necessary  to  remove  a  laige  miss  distance  when  the  RV  is  re¬ 
solved.  If  it  is  a  tumbling  missile,  the  end  containing  the 
warhead  must  be  identified  before  the  kill  point  on  the  war¬ 
head  can  be  determined.  Measurable  characteristics  of  the 
taiget  image  include  centroid,  principal  axis,  length,  width, 
area,  and  intensity.  Data  quality  can  be  improved  by  remov¬ 
ing  blurring  and  by  c(mbining  consecutive  images  to  obtain 
more  accurate  taiget  intensity  and  shape  information. 

Aimpoints  are  easily  calculated  if  the  target  attitude  is 
known.  They  lie  on  the  principal  axis  of  the  taiget  image,  a 
certain  portion  of  the  way  from  the  nose  end  to  the  tail  end. 
The  principal  axis,  the  certain  portion  of  distance,  and  the 
nose  versus  tail  identification  are  based  on  1)  the  rotation 
angle  of  the  target’s  longitudinal  axis’  component  in  the  fo¬ 
cal  plane,  and  2)  the  aspect  angle  between  the  target’s 
longitudinal  axis’  and  a  normal  to  the  focal  plane.  These  two 
angles  define  the  taiget  attitude. 

A  history  of  rotation  and  aspect  angles  is  necessary 
when  the  taiget  attains  an  attitude  that  is  difficult  to  estimate. 
For  example,  if  the  aspect  angle  becomes  zero  (head-on 
case),  then  the  rotation  angle  is  undefined  and  is  not  required 
for  aimpoint  estimation.  Since  a  symmetrical  target’s  rota¬ 
tional  motion  constrains  the  aspect  and  rotation  angles,  they 
are  not  independent  of  one  another  and  their  histories  can  be 
advantageously  estimated  together  in  a  smoothing  filter. 

In  summary,  a  fundamental  problem  is  to  determine  tar¬ 
get  attitude  and  target  type  so  as  to  hit  a  precise  kill  point. 
Solving  the  problems  of  increasing  target  resolution  and  esti¬ 
mating  target  attitude  enables  selection  of  the  aimpoint  and 
prediction  of  its  location  at  impact. 

4  Detection.  Selection.  Aimpoint  Algorithms 

The  phase  transition  of  the  algorithms  is  depicted  in 
Figure  1 .  Objects  are  detected  and  tracked  to  increase  the  in¬ 
formation  available  for  discriminating  the  taiget.  After  the 
taiget  is  selected,  the  homing  phase  starts,  and  the  centroid 
is  provided  to  guidance.  When  the  target  image  is  large 
enough  to  identify  the  warhead,  transition  to  the  aimpoint 
phase  results  in  providing  the  kill  point  location  to  guidance. 


5-23 


UNCLASSIFIED 


UNCLASSIFIED 


FPALJ 


Pre 

amp 


Focal  Plane 
Electronics 


Analog  Og 


DSP 


II 

■ 

I 

1511 

1 

■1 

■ 

1 

wism 

1 

8 

■ 

16 


To 

GN&C 


Analog  r,  skim, 
capacitance 


Digital  r,  skim, 
capacitance 


Figure  2.  Variables  set  during  calibration. 


4.1  Detection  Algorithms 

Detection  at  the  earliest  possible  time  requires  analog 
and/or  digital  integration  to  enhance  SNR.  Analog  integra¬ 
tion  time  and  detector  readout  capability  are  increased  by  a 
skim  voltage  that  removes  electrons  due  to  photons  from 
telescope  self  emission.  Capacitor  size  can  be  varied  to  in¬ 
crease  the  size  of  the  readout.  Digital  calibration  may  not  be 
required  if  detector  linearity  is  good,  and  if  off-focal  plane 
integration  accounts  for  background  mtensity,  which  in¬ 
cludes  pattern  noise. 


4.1.1  Calibration  Figure2  shows  the  variables  that  can 
be  set  during  calibration.  The  variables  integration  time,  r, 
skim  voltage,  skim,  and  global  offset,  Og ,  interact.  An  itera¬ 
tive  procedure  optimizes  skim  subject  to  variations  in  the 
skim  control  field  effect  transistors  (FETs).  If  digital  calibra¬ 
tion  is  used,  standard  statistically  based  procedures  determine 
a  digital  gain  and  offset  and  Oij  for  detector  ij,  where  i 
and  j  are  the  row  and  column  numbers,  respectively. 


4.12  Off-focal  Plane  Integration 

Off-focal  plane  integration  sums  frames  to  produce 
master  frames  at  a  lower  frame  rate  [2].  Sensor  motion 
causes  subframes  to  have  different  viewing  directions,  so 
each  subframe  has  its  own  spatial  offset  applied  to  compen¬ 
sate  for  that  motion  before  the  frame  is  added  to  the  master 
frame.  Each  pixel  of  the  resulting  master  frame  contains  the 
signal  from  a  particular  viewing  direction.  The  FOR  is  repre¬ 


sented  by  an  inertially  fixed  2D  array  of  pixels  and  the  output 
from  each  detector  is  added  to  the  pixel  that  has  the  same 
viewing  direction  as  the  detector.  A  variation  fixes  the  FOR 
relative  to  a  target  rather  than  to  inertial  space. 

The  summing  increases  SNR  with  respect  to  the  white 
noise  component,  but  the  background  must  still  be  removed 
from  the  master  frame.  Off-focal  plane  integration  assumes 
that  the  output  of  a  given  detector  in  a  subframe  remains  es¬ 
sentially  free  of  signal  while  the  subframes  are  being 
accumulated.  This  output  is  estimated  for  each  detector  by 
taking  a  temporal  average  of  the  detector’s  readout  values 
during  the  accumulation  of  a  master  frame  and  determines 
a  background  value  for  that  detector.  The  background  is  then 
repeatedly  subtracted  from  the  master  frame  using  the  same 
motion-compensating  spatial  offsets  as  when  the  subframes 
were  accumulated. 

Off-focal  plane  integration  algorithms  estimate  and  re¬ 
move  background  due  to  the  scene,  sensor  self  emission,  and 
Ij^f  noise.  They  increase  SNR  by  digital  integration  leading 
to  earlier  detection  of  objects.  The  sensor  response  in  re¬ 
gions  of  interest  is  removed  to  the  greatest  degree  possible 
by  deconvolving  the  sensor  response  function  from  the 
image. 

Figure  3  shows  the  result  of  off-focal  plane  integration. 
A  single  frame  shows  a  strong  signal,  but  nothing  else  other 
than  noise.  After  off-focal  plane  integration,  three  low  in¬ 
tensity  signals  are  also  present.  Note  that  an  offset  evident 
in  Figure  3  (a)  is  removed  by  the  background  removal  as 
shown  in  Figure  3  (b). 

4.2  Selection  Algorithms 

Selection  algorithms  are  required  for  an  operational 
KKV  to  discriminate  a  target  from  other  objects  in  the  FOV. 
Near  term  objects  that  must  be  discriminated  include  pieces 
of  solid  debris  from  the  booster,  the  booster  itself  if  it  is  sepa¬ 
rated  from  the  warhead,  and  booster  fragments.  Eventually, 
boosters  may  include  explosives  to  break  it  up  along  fracture 


Figure  3.  Off-focal  plane  integration,  (a)  Single  frame  and  (b)  master  frame. 
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lines  introduced  to  control  the  size  of  the  booster  fragments. 
The  fragments  then  must  be  discriminated  from  the  warhead. 
Far  term  discrimination  problems  involve  sophisticated 
threats  that  include  decoys,  balloons,  aerosols,  chaff,  etc.  in 
the  objects  deployed  with  the  RV.  All  of  these  objects  exhibit 
thermal,  thermal  rate  of  change,  emissivity  area,  and  intensi¬ 
ty  rate  of  change  that  enable  discrimination  of  these 
nonlethal  objects  from  the  warhead. 

4.2.1  Algorithm  Interaction  Figure  4  illustrates  the 
interaction  between  off~focal  plane  integration  (OFPI)  and 
target  selection.  The  features  extraction  function  of  Figure  4 
identifies  discrimination  features  based  upon  repeated  angu¬ 
lar  position  and  intensity  measurements,  used  for  object 
differentiation  by  classification  algorithms.  Table  1  illus¬ 
trates  typical  features.  It  is  clear  from  this  table  that  objects 
can  not  simply  be  discriminated  using  a  single  feature.  In 
addition  accurate  estimation  of  an  object’s  temperature  re¬ 
quires  multi” wavelength  measurements. 

Classification  algorithms  address  the  discrimination 
problem  by  using  feature  vectors  of  the  objects  in  the  seek¬ 
er’s  FOR.  These  vectors  are  compared  in  the  classifier  using 
rules  developed  from  training  data  to  select  the  most  prob¬ 
able  target.  This  selection  can  be  delayed  only  as  long  as  is 
consistent  with  the  allowable  surface  defining  the  reachable 
targets. 

4.2.2  Tracking,  feature  extraction,  classification  A 
tracking  algorithm  determines  a  trajectory  for  each  object  so 


that  the  master  and/or  frame  feature  data  is  associated  with 
the  correct  object.  The  features,  as  has  been  mentioned,  are 
based  on  object  motion,  intensity,  and  motion/intensity  rates. 
The  features  are  estimated  with  the  greatest  accuracy  pos¬ 
sible  by  deconvolving  the  sensor  response  to  determine 
trajectories  to  subpixel  accuracy,  and  by  estimating  the  am¬ 
plitude  with  the  greatest  precision  possible. 

Classification  requires  features  or  discriminants  to  dis¬ 
tinguish  objects  including  celestial  bodies,  missiles  with 
attached  warheads,  reentry  vehicles,  shrouds,  tanks,  tank 
fragments,  balloons,  chaff,  aerosols,  decoys,  and  other 
associated  objects.  Qassification  techniques  based  on  phe¬ 
nomenology  discriminants  have  been  studied  in  the  past  [3]. 
Other  discriminants  based  on  multiple  observations  to  iden¬ 
tify  kinematics  of  an  object  were  also  investigated. 

Once  the  feature  vector  consisting  of  measurable  dis¬ 
criminants  is  obtained,  the  type  of  object  can  be  identified  by 
classification  process.  Classification  algorithms  can  be  rela¬ 
tively  simple  for  near  term  threats,  but  become  increasingly 
sophisticated  for  far  term  threats.  A  near  term  classifier 
based  on  single  color  focal  plane  data  uses  object  motion  and 
intensity  and  their  rates. 

4.3  Aimpoint  Algorithms 

The  aimpoint  algorithms  estimate  the  desired  impact 
point  based  on  the  sensor’s  target  image  to  within  a  small 
angle  or,  equivalently,  to  within  a  small  number  of  pixels  [5]. 
At  longer  ranges,  a  centroid  of  the  target  image  is  sufficiently 


Table  1.  Typical  object  features  which  allow  target  selection. 
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close  to  the  position  of  the  target’s  desired  impact  point  At 
closer  ranges,  when  the  target’s  attitude  becomes  known, 
then  the  position  of  the  desired  impact  point  is  calculated. 

The  important  parameters  for  aimpoint  determination 
are  target  rotation  and  aspect  relative  to  the  KKV  sensor 
LOS,  (see  [4],  [5]  for  coordinate  frame  description),  the  nose 
end  of  the  target,  and  target  type.  Nose  end  identification  is 
particularly  important  if  the  target  is  a  complete  missile. 
Wiih  the  rotation  angle  known,  aimpoints  can  be  estimated 
with  sufficient  accuracy  while  the  aspect  angle  is  still  being 
determined.  Target  range  is  estimated  from  changes  in  the 
target  image’s  width,  area,  and  intensity.  Range  together 
with  length  of  the  target  provide  the  target  aspect  angle. 

4.3.1  Target  Image  Measurable  Characteristics  As 
mentioned  earlier,  measurable  characteristics  of  the  target 
image  include:  centroid,  principal  axis,  length,  width,  area, 
and  intensity.  Consecutive  images  are  combined  to  improve 
the  quality  of  the  image  used  to  measure  target  characteris¬ 
tics.  This  is  done  by  aligning  the  target  image  centroid 
positions  and  interpolating  each  image  onto  a  common  grid 
covering  the  target  area.  Bilinear  interpolation  is  used  for  a 
value  at  a  grid  position  from  the  values  of  pixels  surrounding 
that  position.  The  measurements  are  then  made  on  this  grid. 

The  centroid  of  the  target  image  consists  of  the  first 
moments  of  pixel  intensities  over  a  threshold.  The  principal 
axis  orientation  is  calculated  directly  from  the  second 
moments  of  those  pixel  intensities.  The  length  of  the  target 
image  is  obtained  by  evaluating  intensity  along  the  principal 
axis  and  applying  a  threshold  to  determine  the  end  points  of 
the  axis.  Similarly,  the  width  of  the  target  image  is  obtained 
by  intensity  thresholding  along  normals  to  the  principal  axis. 
The  normals  are  placed  at  predetermined  positions  along  the 
length  of  the  axis.  For  a  missile  target,  widths  are  calculated 
near  the  midpoint  and  averaged  together.  For  an  RV  target, 
the  largest  width  along  the  entire  axis  is  used. 

The  area  of  the  target  image  is  the  number  of  its  pixels 
over  threshold,  and  the  intensity  is  the  sum  of  its  pixel  inten¬ 
sities.  Also,  the  intensity  at  a  specific  position  in  a  target 
image  is  obtained  by  bilinear  interpolation  of  the  values  of 
the  four  surrounding  pixels. 

Target  rotation  angle  is  the  orientation  of  the  target 
image’s  principal  axis  with  respect  to  a  focal  plane  axis.  The 
aspect  angle  is  estimated  from  the  target  image  length  to¬ 
gether  wifli  either  the  target’s  range  or  the  target  image 
width. 

4.3.2  Nose  End  Identification  An  example  of  an  algo¬ 
rithm  for  nose  end  identification  is  an  estimator  of  the  width 
of  the  target  image  at  both  ends  of  the  principal  axis.  The 


width  at  each  end  is  the  average  of  many  widths  measured 
along  a  fraction  of  the  length.  The  two  end  widths  are  aver¬ 
aged  over  consecutive  frames  and  the  narrower  end  is 
designated  the  nose  of  the  target. 

5  Multi-microprocessor  Implementation 

This  section  describes  real-time  algorithm  imple¬ 
mentation  using  four  TMS320C30  microprocessor  devices. 
Selection  algorithms  are  for  a  near  term  threat,  so  the  prima¬ 
ry  processing  difficulty  is  the  OFPI. 

5.1  Partitioning  for  Real-Time  Implementation 

In  a  focal  plane  output  partitionmg  that  accommodates 
a  256  X  256  pixel  focal  plane  read  at  120  Hz,  each  processor 
is  given  a  region  of  the  input  image  with  the  data  allotted  in 
the  order  read.  The  regions  may  overlap.  For  OFPI  each  mi¬ 
croprocessor  is  given  a  horizontal  strip  comprising 
one-quarter  of  the  image,  as  shown  in  Figure  5. 

5.2  Hardware  Configuration 

Figure  6  illustrates  the  microprocessor  hardware  con¬ 
nections.  The  field  programmable  gate  arrays  (FPGAs) 
control  movement  of  the  pixels  to  the  first  in,  first  out  (FIFO) 
devices.  The  shared  memory  allows  for  tight  coupling 
among  processors,  and  for  data  distribution. 

The  shared  memory  bus  bandwidth  is  insufficient  for 
transmitting  entire  images  on  a  contiguous  basis.  Therefore, 
each  microprocessor  processes  and  summarizes  its  portion 
of  the  image.  The  summary  information  is  passed  to  shared 
memory  and  integrated  by  higher  level  algorithms. 
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Figure  5.  Off-focal  plane  integration  timing  diagram. 


Figure  6.  Programmable  multiprocessor  configuration  to  process  algorithms. 
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